(1) Field of the Invention
This invention relates to a rear-view monitor for use in vehicles and, more specifically, to a rear-view monitor that warns a driver to respond to an vehicle approaching from the rear with regard to one's own vehicle by taking images of road in the rear of one's own vehicle with an image pickup device such as a video camera mounted on one's own vehicle such as a motor vehicle and then, by detecting a vehicle approaching from the rear using road images thus obtained.
(2) Description of the Related Art
Several rear-view monitoring and warning systems have hitherto been proposed in the past, for example, Japanese Patent Publication No. 2641562, Japanese Patent Application Laid-Open No. H6-107096 and H7-50769. The systems are capable of monitoring the rear road in order to prevent a possible collision with an overtaking vehicle from occuring by correctly sizing a situation of the overtaking vehicle traveling an adjacent either right or left traffic lane and then, warning a driver of possibility of collision with such overtaking vehicle traveling the adjacent lane when a driver is about to chang the traveling lane.
According to the proposed systems, in order to detect an overtaking vehicle traveling the adjacent lane quickly and securely without performing an excessive image processing, lanes on the road are distinguished by recognizing white lines, i.e. lane dividing lines, using road images taken by a video camera, and a monitoring range is set to the adjacent lane in accordance with this discrimination so that an amount of image processing for detecting an approaching vehicle within an adjacent lane, i.e. a monitoring range set up in the road images taken by a video camera, is reduced.
Apparatuses that employ a detection system by optical flow are disclosed in Japanese Patent Application Laid-Open No. H6-107096 and H7-50769, features of which are explained with reference to FIGS. 7 and 8 in the following.
FIG. 7 is a block diagram illustrating an assembly of a conventional rear-view monitoring and warning system for vehicles, which contains an image pickup unit 10 as an image pickup device such as a video camera 11, a data processing unit 30 as a computing system, and a speaker 42 as a warning means.
The image pickup unit 10 is mounted at the rear side of a vehicle such as on a trunk lid so as to take road images of the rear with regard to the traveling vehicle. The data processing unit 30 contains a CPU 31 as a central processing unit operating in accordance with an operation program, a ROM 32 for memorizing the operation program of the CPU 31 and predetermined settings, and a RAM 33 for provisionally memorizing data required upon computation by the CPU 31. The speaker 42 mounted on the vehicle warns a driver of a danger by sounding or warning in accordance with a signal from the CPU 31 in the data processing unit 30.
FIGS. 8A and 8B illustrate a change in images taken by the image pickup unit 10 mounted at the rear portion of the vehicle. FIG. 8A is an image taken at a time t and FIG. 8B at a time t+.DELTA.t. In each figure, shown are an overtaking vehicle 200 traveling in the rear with regard to one's own vehicle, a traffic sign 300 disposed along a road 500, and a building 400 along the road 500.
Supposing that one's own vehicle is traveling straight along a flat road, with time passing by, i.e. accompanied by the vehicle traveling, the traffic sign 300 and building 400 are relatively leaving from the vehicle, resulting in that the images of the traffic sign 300 and building 400 become small. That is, in the figures, images of the traffic sign 300 and building 400 taken at the time t shown in FIG. 8A is smaller than those taken at the time t+.DELTA.t shown in FIG. 8B.
In the following, the optical flows are explained with reference to these figures.
In these figures, to be considered are a plurality of marked points selected in the images taken at the time t (see FIG. 8A) such as: marked points 201a and 202a for the overtaking vehicle 200; 301a, 302a and 303a for the traffic sign 300; and 401a and 402a for the building 400. Similarly, to be considered are marked points such as: 201b and 202b for the overtaking vehicle 200; 301b, 302b and 303b for the traffic sign 300; and 401b and 402b for the building 400 in the images taken at the time t+.DELTA.t (see FIG. 8B). Then, each combination between corresponding marked points, for example 201a and 201b, gives a velocity vector such as 201F, 202F, 301F, 302F, 303F, 401F and 402F as shown in 8C. These velocity vectors are defined as the optical flows.
Here, it can be understood that the optical flow radially appears from a focus of expansion (hereinafter, FOE) defined as an infinitely far point or a vanishing point in the figures. When one's own vehicle is traveling straight, the FOE corresponds to just the opposite direction to which the vehicle is traveling.
When one's own vehicle is traveling, an optical flow of an object going away from the vehicle is a convergent vector heading toward the FOE, while an optical flow of an object approaching the vehicle is a divergent vector away from the FOE. Accordingly, the optical flows 201F and 202F (shown in FIG. 8C) are divergent vectors, indicating that the vehicle 200 is approaching one's own vehicle, in other wards, that the vehicle 200 is traveling faster than one's own vehicle.
Regarding a size of the optical flow, the size is large when a difference in velocity between one's own vehicle and an object is large and also when a distance therebetween is short. In the following, this matter is explained with reference to the attached drawings.
FIG. 9 illustrates an optical arrangement of the image pickup unit 10, in which 11a is a lens of the video camera in the image pickup unit 10, 11b is an image plane of the video camera, f is a distance between the lens 11a and the image plane 11b, P (X.sub.a, Y.sub.a, Z.sub.a) is an arbitrary point of the overtaking vehicle, and p (Xa, ya) is a point corresponding to the point P on the image plane 11b.
On this occasion, the following formula is given on the basis of similar figures in triangles: EQU X.sub.a =f.multidot.X.sub.a /Z.sub.a (1)
Transforming the formula (1) and then, differentiating with respect to time gives the following formula (2): EQU X.sub.a '=(.DELTA.x.sub.a /.DELTA.t.multidot.Z.sub.a +x.sub.a.multidot.Z.sub.a ')/f (2)
A x-component u of an optical flow is given by the following formula (3): EQU u=.DELTA.x.sub.a /.DELTA.t (3)
Hence, the following formula (4) is derived from the formula (3): EQU Z.sub.a =(f.multidot.X.sub.a '-x.sub.a.multidot.Z.sub.a ')/u (4)
Here, Z.sub.a ' denotes a difference in velocity between the overtaking vehicle (200 in FIG. 8) traveling the same lane or the adjacent lane and one's own vehicle on which the image pickup unit 10 is mounted. Assuming this difference in velocity to be -.alpha., the formula (4) is expressed by the following formula (5): EQU Z.sub.a =(f.multidot.X.sub.a '+x.sub.a.multidot..alpha.)/u (5)
Hence, the x-component u of the optical flow, is expressed by the following formula (6): EQU u=(f.multidot.X.sub.a '+x.sub.a.multidot..alpha.)/Z.sub.a (6)
By the way, Y.sub.a, i.e. Y-coordinate of the point P, can be derived in the similar way.
Consequently, according to the formula (6), when Z is small, i.e. a distance between one's own vehicle and the overtaking vehicle 200 traveling the same lane or the adjacent lane is short, or when a is large, i.e. the difference in velocity between one's own vehicle and the overtaking vehicle 200 is large, an x-component of the optical flow becomes large. These relations are the same with respect to the Y-direction.
Accordingly, the size of the optical flow becomes large when the distance between one's own vehicle and the overtaking vehicle 200 is short, and when the difference in velocity between the both vehicles is large, resulting in that the direction of the optical flow diverges from the FOE. In this case, the larger the size of the optical flow, the larger a degree of danger of the collision.
The data processing unit 30 recognizes a situation, in which the optical flow is the divergent vector as mentioned above and also the size of the optical flow is large, caused by either or both situations as follows: one situation that the object is in the vicinity of one's own vehicle and another situation that the object is approaching one's own vehicle with higher speed than that of one's own vehicle, and the data processing unit 30 judges that the degree of danger is high. Thus, the data processing unit 30 warns a driver of the danger via the speaker when the unit judges that the degree of danger is high.
By such repeated data processings with respect to all the points on the image taken at the time t, optical flows covering the whole image can be obtained, and each degree of danger corresponding to each object is determined. Then, an attention is given to a driver by warning sounds and the like according to the degree of danger determined, resulting in giving a complement to a limited recognition by a human being and prevention of an actual traffic accident from occurring or being out of dangerous situation which might otherwise develop into a serious accident.
As shown in FIG. 10, in a conventional art, a detection of white lines of a lane, along which one's own vehicle is traveling a straight six-lane expressway, brings about a discrimination between said lane and adjacent lanes and a determination of range to be monitored, aiming such that a processing time for monitoring of unnecessary field of vision can be saved and a high-speed processing can be achieved. An extension of thus detected white lines determines a FOE point and the overtaking vehicle 200 is detected by determining optical flows developed radially from said FOE point regarding a range of one's lane and adjacent lanes. Since a necessary recognition in the system is formed on the basis of the size of the optical flows, a degree of danger regarding the overtaking vehicle 200 traveling the rear or adjacent lanes is automatically judged and advantageously, an extra speedometer is not necessary.
Among currently proposed detection methods of the optical flow, a correlation method is nearly on a level of practical use. The correlation method has a disadvantage of enormous amount of computation since the method contains a search for corresponding points of window (pixel) with respect to all circumferential areas and a computation of correlation values. On the other hand, the method has an advantage of finding relatively correct optical flow regarding a complicated image, for which the present invention has as a subject.
As mentioned above, according to the correlation method, when optical flows are to be found regarding an image at the time t, searchs in all directions are necessary to find out which pixel among all the pixels in an image at the time t corresponds to which pixel in an image at the time t-.DELTA.t, causing enormous amount of computation as well as a possible error in required response. Hence, it is considered that limiting a monitoring range might solve problems such as long precessing time and low detection accuracy.
As for the aforementioned type of rear-view monitoring system for use in vehicles, from an image taken by a video camera, a large image of another vehicle in the vicinity of one's own vehicle can be obtained compared with that far from one's own vehicle, but only a small image of the other vehicle far from one's own vehicle can be obtained. Therefore, an image of low resolution is good enough in the vicinity of one's own vehicle, but a distant image with low resolution makes it difficult to catch a behavior of another vehicle precisely by an image processing and consequently, it is preferable that images with high resolution as much as possible can be obtained regarding distant images.
In addition, taking a case of lane change into consideration, it is preferable to use a wide-angle video camera to monitor adjacent lanes as close as possible to one's own vehicle. However, the use of the wide-angle video camera makes the monitoring range wider, resulting in deterioration in the resolution and increase in number of image data to be processed, causing problems such as long precessing time and low detection accuracy.
Thus, regarding this types of monitoring system, in which monitoring of the far and near distances are simultaneously needed, there are above-mentioned contradictory requirements and therefore, a measure to meet with these requirements has been desirable.